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Abstract 

This  study  assessed  substance  use  and  related  factors  among  sixth  through  twelfth  grade 
participants  in  a  youth  development  program.  Comparisons  between  participants  ( N  =  361)  and  a 
national  youth  sample  indicated  that  participants  had  lower  30-day  and  lifetime  alcohol  use, 
lower  30-day  and  lifetime  prevalence  of  any  illicit  drug  use,  and  higher  drug-averse  attitudes 
than  the  national  sample.  Comparisons  between  participants  and  new  enrollees  ( N  =  145) 
indicated  higher  drug  knowledge  among  participants.  The  study  also  showed  a  short-term 
increase  in  self-esteem  among  new  enrollees,  no  changes  in  substance  use,  and  an  unexpected 
negative  effect  for  drug  knowledge  after  3  months  in  the  program.  Overall,  the  program  had  a 
minimal  impact  on  participants’  drug  use  and  related  factors.  Recommendations  for  program 
improvements  are  discussed. 
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Substance  Use  Prevention  in  a  Youth  Development  Program 
Youth  substance  abuse  is  a  critical  national  concern,  with  devastating  medical,  social,  and 
economic  costs  (Centers  for  Disease  Control  and  Prevention,  2008;  Johnston,  O’Malley, 
Bachman,  &  Schulenberg,  2007;  Substance  Abuse  and  Mental  Health  Services  Administration 
[SAMHSA],  2008).  Poor  academic  perfonnance,  school  attrition,  infectious  diseases,  violence, 
and  other  behavioral  problems  are  just  some  of  the  hannful  consequences  of  youth  drug  abuse 
and  addiction,  making  prevention  a  key  issue  (Huizinga,  Loeber,  &  Thornberry,  1993;  National 
Institute  on  Drug  Abuse  [NIDA],  2006;  Newcomb  &  Bentler,  1988). 

Positive  youth  development  programs  are  receiving  much  interest  as  a  method  for 
reducing  the  risk  of  youth  drug  abuse  (Catalano,  Berglund,  Ryan,  Lonczak,  &  Hawkins,  2002). 
These  types  of  programs  seek  to  promote  a  positive  developmental  pathway  for  youth,  and 
prevent  the  occurrence  of  problems,  by  promoting  social,  emotional,  and  behavioral  competence 
or  skills,  fostering  resilience,  self-efficacy  and  social  norms,  as  well  as  providing  drug  prevention 
education.  Youth  development  programs  often  use  a  combination  of  drug  abuse  prevention 
approaches  that  may  include  social  influence  and  skills  training,  modifying  risk  and  protective 
factors,  and  traditional  education  and  information  dissemination. 

The  effectiveness  of  youth  development  programs  in  changing  drug-related  behaviors, 
attitudes,  and  knowledge  has  varied.  Several  school-based  programs,  or  programs  with  school- 
based  components,  have  demonstrated  positive  behavioral  effects  among  youth,  such  as 
decreased  alcohol,  cigarette,  or  marijuana  use  (Botvin,  Baker,  Dusenbury,  Tortu,  &  Botvin, 

1990;  Ellickson,  Bell,  &  Harrison,  1993;  Perry,  Williams,  Veblen-Mortenson,  Toomey,  Komro, 
Anstine,  et  al.,  1996)  and  lower  overall  drug  use  (LoSciuto,  Freeman,  Harrington,  Altman,  & 
Lanphear,  1997).  Participation  in  less-structured  prevention  programs,  such  as  regular  attendance 
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at  the  Boys  and  Girls  Clubs  of  America,  and  participation  in  the  Big  Brothers  Big  Sisters 
mentoring  program,  have  been  shown  to  be  associated  with  lower  drug  use  (Anderson-Butcher, 
Newsome,  &  Ferrari,  2003;  Tierney,  Grossman,  &  Resch,  1995).  In  addition  to  positive  results,  a 
few  youth  development  programs  have  shown  unanticipated  negative  effects  such  as  increased 
drug-related  attitudes  (LoSciuto  et  ah,  1997)  and  increased  cigarette  use  among  baseline  smokers 
(Ellickson  &  Bell,  1990).  In  some  studies  that  also  targeted  psychosocial  factors  related  to 
substance  use,  such  as  self-efficacy  and  self-esteem,  no  effects  were  found  for  these  factors 
(LoSciuto  et  al.,  1997;  Perry  et  ah,  1996).  It  is  difficult  to  draw  strong  conclusions  about  the 
effectiveness  of  youth  development  programs  from  existing  research  due  to  the  variability  in 
measures  and  reported  findings.  This  study  addressed  the  need  for  additional  research  in  this  area 
regarding  the  effects  of  a  youth  development  program  on  substance  use  prevention. 

The  objectives  of  this  study  were  to  (1)  evaluate  the  effectiveness  of  an  existing  youth 
development  program  on  drug  use  by  comparing  self-reported  drug  use,  drug-related  attitudes, 
and  knowledge,  and  positive  character  traits  of  program  participants  with  two  peer  groups;  and 
(2)  assess  short-tenn  changes  in  drug  use  and  related  factors  among  program  participants. 

Methods 

Study  Design 

This  study  consisted  of  two  methodological  approaches.  The  first  approach  was  to 
compare  self-reported  drug  use,  attitudes,  knowledge,  and  psychosocial  characteristics  among 
program  participants  with  two  comparison  groups:  a  national  sample  of  adolescents  from  a  large, 
in-school  survey  of  drug  use,  and  children  who  recently  enrolled  in  the  youth  development 
program  but  had  not  yet  been  exposed  to  the  program’s  routine  drug  prevention  education.  The 


Y outh  Substance  Abuse  Prevention  5 


second  approach  was  a  pretest-posttest  study  with  new  program  participants  to  assess  short-term 
changes  in  drug  use,  attitudes,  knowledge,  and  positive  character  traits  over  a  3-month  period. 
Program 

The  program  evaluated  in  this  study  is  a  youth  development  program  that  focuses  on 
character  building,  teamwork,  leadership,  fitness  and  drug  prevention  education.  This  national 
program  is  organized  into  local  community-based  programs  run  by  adult  volunteers.  Participants 
meet  with  their  local  program  once  or  twice  per  week  year-round.  The  content  of  the  drug 
prevention  education  focuses  on  problem  solving  and  resisting  peer  pressure,  drug  facts  and  the 
effect  of  drugs  on  the  body,  social  and  media  influences,  drug  laws,  alternative  activities,  and 
being  role  models  for  other  youth.  However,  the  specific  content  of  the  lessons  and  its 
implementation  is  determined  by  the  individual  local  programs.  Common  strategies  used  by  the 
program  for  drug  prevention  are  providing  structured  activities,  group  educational  presentations, 
and  other  infonnation  dissemination. 

Participants 

The  study  participants  were  boys  and  girls  in  the  sixth  through  twelfth  grades  who  were 
currently  enrolled  in  the  youth  development  program  as  regular  members  or  new  enrollees  at 
units  across  the  United  States.  Regular  members  were  participants  who  had  completed  the 
orientation  process  and  were  active  members  of  the  program.  New  enrollees  were  participants 
who  had  not  yet  completed  the  orientation  process  at  the  time  of  the  survey.  Hence,  the  new 
enrollee  comparison  group  was  enrolled  in  the  program  but  not  yet  exposed  to  the  intervention. 

The  national  comparison  group  was  obtained  from  the  Monitoring  the  Future  (MTF) 
study,  which  is  an  in-school  survey  of  drug  use  based  on  probability  samples  of  adolescents 
(http://monitoringthefuture.org/).  A  comparison  data  file  of  youth  in  eighth,  tenth,  and  twelfth 
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grades  was  created  from  the  publicly  available  MTF  datasets  from  2006  (Substance  Abuse  and 
Mental  Health  Data  Archive  [SAMHDA]),  the  most  recent  data  available  at  the  time  of  the 
study. 

Procedures 

A  random  sampling  procedure,  stratified  by  program  size,  was  used  to  identify  90  local 
programs  that  were  targeted  for  the  study.  Parents  of  program  participants  were  made  aware  of 
the  opportunity  for  their  child  to  take  part  in  the  study  via  an  online  announcement  posted  on  the 
program’s  Web  site  and  flyers  that  were  sent  to  the  targeted  local  program  leaders.  Parents 
provided  their  written  infonned  consent  via  regular  mail  before  the  survey  was  administered. 
Youth  provided  their  assent  just  prior  to  completing  the  survey.  The  investigators  administered 
the  survey  in-person  to  the  larger  programs  at  their  regularly  scheduled  meetings  and  password- 
protected,  online  survey  access  was  made  available  for  all  other  participants  with  Internet  access. 
The  research  staff  emphasized  the  voluntary  nature  of  the  survey  and  the  confidentiality  of  the 
data  collection.  The  Flesch-Kincaid  reading  level  of  the  survey  was  6.9,  corresponding  to  an 
approximately  seventh-grade  reading  level;  however,  the  research  staff  helped  participants  who 
experienced  difficulty  understanding  the  survey  questions.  The  youth  took  approximately  30  to 
45  minutes  to  complete  the  survey.  For  participants  who  completed  the  survey  online,  the 
opening  screen  of  the  survey  reminded  the  parent  and  the  youth  that  the  survey  was  to  be  filled 
out  by  the  youth  in  complete  privacy. 

Measures 

The  survey  queried  participants  on  demographics,  drug  behaviors,  drug-related  attitudes, 
drug  knowledge,  psychosocial  characteristics,  and  other  items.  In  order  to  facilitate  comparisons 
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with  the  national  sample,  the  alcohol,  tobacco,  and  drug  use;  attitudes;  and  self-efficacy  items 
mirrored  those  in  the  national  MTF  study  (Bachman,  Johnston,  &  O’Malley,  2008). 

Demographics .  Gender,  race,  age,  and  grade  were  assessed  in  the  survey,  as  well  as 
length  of  time  in  the  program. 

Alcohol  and  tobacco  use.  Cigarette  smoking  was  assessed  by  asking:  “Have  you  ever 
smoked  cigarettes?”  and  “During  the  past  30  days,  about  how  many  cigarettes  have  you  smoked 
per  day?”  Thirty-day  use  was  defined  as  any  use  of  cigarettes  during  that  period  and  was 
reported  among  only  those  respondents  who  had  ever  smoked  cigarettes.  Alcohol  use  was 
measured  similarly  by  querying  lifetime  use,  as  well  as  use  within  the  last  30  days.  Thirty-day 
alcohol  use  was  reported  among  only  those  respondents  who  had  ever  had  any  alcoholic 
beverage. 

Drug  use.  Participants  were  queried  about  their  drug  use  levels  for  various  drugs  using  a 
standard  set  of  two  questions:  “On  how  many  occasions  (if  any)  have  you  used  [specific 
drug] . .  .(a)  in  your  lifetime?  and  (b)  during  the  last  30  days?”  Both  questions  were  answered 
using  the  same  response  scale:  0  occasions,  1-2,  3-5,  6-9,  10-19,  20-39,  and  40  or  more 
occasions.  Illicit  drug  use  was  defined  as  the  use  of  marijuana/hashish,  LSD,  hallucinogens  other 
than  LSD,  cocaine,  amphetamines,  methamphetamine,  crystal  methamphetamine,  sedatives, 
tranquilizers,  narcotics  other  than  heroin,  steroids  or  MDMA  (ecstasy).  For  the 
psychotherapeutic  drugs  (amphetamines,  sedatives,  tranquilizers,  and  narcotics  other  than  heroin) 
and  anabolic  steroids,  respondents  were  instructed  to  include  only  use  “on  your  own — that  is, 
without  a  doctor  telling  you  to  take  them.”  Participants  who  answered  “1-2”  or  more  occasions 
of  use  for  one  or  more  illicit  drug  items  for  the  last  30  days,  regardless  of  missing  data,  were 
counted  as  current,  any  illicit  drug  users.  Participants  who  reported  “0”  occasions  for  all  drug 
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items  were  counted  as  non-illicit  drug  users;  those  with  missing  data  were  excluded.  Missing 
“past  30  days”  drug  values  for  cases  with  lifetime  use  of  “0”  on  that  particular  drug,  were 
imputed  to  have  a  past  30  days  value  of  “0.” 

Drug-related  attitudes.  Attitudes  and  aspects  of  the  immediate  social  environment  that 
may  have  affected  a  participant’s  use  of  drugs  were  measured  including  the  extent  of  exposure  to 
friends’  use,  personal  disapproval  of  drug  use,  perceived  risk  of  harmfulness  of  various  drugs, 
and  perceived  availability  of  drugs.  Exposure  to  friends’  use  was  measured  with  the  item,  “How 
many  of  your  friends  would  you  estimate?...”  followed  by  “...smoke  cigarettes,”  for  example. 
Response  categories  were,  “None,”  “A  few,”  “Some,”  “Most,”  and  “All.”  Five  items  that  asked 
about  the  use  of  cigarettes,  marijuana,  cocaine,  drinking  alcohol,  and  getting  drunk  were  used  to 
create  an  overall  friends’  drug  use  scale.  The  scale’s  coefficient  alpha  for  the  present  regular 
participant  sample  was  0.88.  Personal  disapproval  of  substance  use  was  measured  by  the 
question,  “Do  you  disapprove  of  people  doing  each  of  the  following?”  followed  by  “trying 
marijuana  once  or  twice,”  for  example.  Response  categories  were,  “Don’t  disapprove,” 
“Disapprove,”  and  “Strongly  disapprove.”  Six  items  that  asked  about  the  use  of  cigarettes, 
marijuana,  cocaine  powder,  “crack”  cocaine,  drinking  alcohol  nearly  every  day,  and  binge 
drinking  were  used  to  create  an  overall  personal  disapproval  of  drug  use  scale.  The  scale’s 
coefficient  alpha  for  the  present  regular  participant  sample  was  0.85.  Perceived  harmfulness  was 
measured  by  the  item,  “How  much  do  you  think  people  risk  hanning  themselves  (physically  or  in 
other  ways),  if  they. . .”  followed  by  “. .  .smoke  one  or  more  packs  of  cigarettes  per  day,”  for 
example.  The  response  categories  were,  “No  risk,”  “Slight  risk,”  “Moderate  risk,”  “Great  risk,” 
and  “Can’t  say,  drug  unfamiliar.”  Seven  items  that  asked  about  the  use  of  cigarettes,  marijuana, 
cocaine  powder,  “crack”  cocaine,  drinking  alcohol  nearly  every  day,  binge  drinking,  and  heroin 
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use  were  used  to  form  an  overall  risk  of  harm  of  drug  use  scale.  The  resulting  coefficient  alpha 
for  this  7-item  scale  was  0.90.  Perceived  availability,  or  access  to  drugs,  was  measured  by  the 
question,  “How  difficult  do  you  think  it  would  be  for  you  to  get  alcohol,  marijuana  (each  asked 
separately),  if  you  wanted  some?”  Response  categories  were:  “Probably  impossible,”  “Very 
difficult,”  “Fairly  difficult,”  “Fairly  easy,”  and  “Very  easy.”  The  two  items  were  used  to  create 
an  access-to-drugs  scale,  with  a  resulting  coefficient  alpha  of  0.80. 

Drug  knowledge.  Drug  knowledge  was  assessed  with  15  multiple-choice  items  about  the 
dangers  and  consequences  of  drug  use,  drug  classifications,  and  common  forms  of  drug 
substances.  The  content  was  derived  from  the  organization’s  guide  books  for  the  youth.  The 
group  of  questions  included  3  tobacco-related  items,  4  alcohol-related  items,  3  marijuana-related 
items,  and  5  items  on  other  drugs.  Overall  drug  knowledge  was  calculated  as  the  number  of 
correctly  answered  items. 

Psychosocial  characteristics.  Psychosocial  factors  included  self-esteem  and  self- 
efficacy/personal  responsibility.  Self-esteem  was  measured  using  12  questions  that  were 
modified  from  the  Rosenberg  Self-Esteem  Scale  (Rosenberg,  1965).  The  items  included 
statements  such  as,  “I  have  a  positive  attitude  toward  myself,”  “On  the  whole,  I’m  satisfied  with 
myself,”  and  “I  enjoy  life  as  much  as  anyone.”  Response  categories  were:  “Disagree,”  “Mostly 
Disagree,”  “Neither,”  “Mostly  Agree,”  and  “Agree.”  Self-efficacy/personal  responsibility  was 
measured  using  the  investigator-developed  10-item  scale  that  asked,  “How  much  do  you  agree  or 
disagree  with  the  following  statements?”  Items  included  statements  such  as,  “I’m  confident  I  can 
avoid  drinking  alcohol,”  “I’m  confident  I  can  set  goals  and  achieve  them,”  and  “I  have  a 
responsibility  to  make  the  world  a  better  place.”  Response  categories  were:  “Strongly  Agree,” 
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“Agree,”  “Disagree,”  and  “Strongly  Disagree.”  The  coefficient  alpha  for  this  10-item  scale  was 
0.85. 

Statistical  Analyses 

It  was  important  for  the  regular  program  participants  to  be  representative  of  the  general 
youth  population  in  order  to  facilitate  the  comparison  with  the  national  sample  of  adolescents. 
Consequently,  we  standardized  the  regular  participant  sample  to  reflect  the  grade,  gender,  and 
race  distribution  of  the  national  sample  by  applying  poststratification  survey  weights.  To 
increase  robustness  and  overall  power  of  the  national  database  comparison,  we  used  all  of  the 
available  2006  MTF  survey  data  (i.e.,  all  available  cases  without  missing  data  for  grade,  gender, 
and  race).  (However,  because  the  national  study  used  multiple  questionnaire  fonns  for  each 
grade  level,  and  because  not  all  of  the  same  questions  were  contained  on  all  forms,  the  number  of 
cases  reported  for  a  particular  item  in  our  comparison  varies.  In  addition,  a  number  of  survey 
items  were  given  only  to  twelfth  graders  in  the  national  study.  For  the  comparison  of  these 
variables,  the  analyses  were  conducted  between  eleventh-  and  twelfth  grade-only  regular 
program  participants  [weighted  only  on  gender  and  race]  and  the  MTF  twelfth  graders.) 

Descriptive  statistics,  including  frequency  and  percentage  distributions,  means,  and 
standard  deviations,  were  computed  to  examine  the  survey  data.  Independent  t  tests  for  means 
and  chi-square  tests  for  proportions  were  conducted  to  examine  differences  in  key  variables  in 
the  weighted  regular  participant  sample  and  the  national  sample,  as  well  as  the  regular 
participants  and  new  enrollees.  Fisher’s  exact  value  tests  were  used  in  comparisons  where  the 
expected  value  of  any  cell  was  less  than  5.  McNemar  tests  and  paired  t  tests  were  perfonned  to 
examine  the  pre-  and  posttest  data.  Statistical  analyses  were  perfonned  using  SPSS  software, 

Rel.  17.0.0,  2008  (SPSS,  Inc.,  Chicago,  IL). 
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Results 

Comparison  Study 

Initial  survey  response  rate.  A  total  of  54  of  the  90  targeted  programs  participated,  for  an 
overall  60.0%  program  response  rate.  Regarding  individual  participants,  we  collected  514 
surveys  from  657  program  members  whose  parents  had  signed  a  consent  fonn,  resulting  in  a 
78.2%  individual  response  rate.  Eight  surveys  had  missing  data  for  the  item  that  indicated 
whether  the  participant  was  a  regular  program  participant  or  a  new  enrollee  and  therefore  were 
not  included  in  the  analyses.  Therefore,  the  final  sample  size  was  N=  506.  This  group  of 
respondents  included  361  regular  program  participants  and  145  new  enrollees. 

The  MTF  national  survey  reported  an  overall  individual  response  rate  that  ranged  from 
82%  to  89%  (Bachman,  Johnston,  O’Malley,  &  Schulenberg,  2006).  The  MTF  comparison 
sample  constructed  for  the  present  study  contained  46,466  respondents  from  the  2006  survey. 

Background  characteristics.  The  program  participants  ranged  in  age  from  10  to  18  years. 
Table  1  shows  that  the  mean  age  of  regular  program  participants  was  13.9  years.  The  majority  of 
regular  and  new  enrollee  participants  were  male.  The  national  sample  had  a  smaller  percentage 
of  males  than  the  regular  participant  sample  ( p  <  .001).  The  majority  of  participants  were  in  the 
sixth,  seventh,  and  eighth  grades.  However,  a  larger  percentage  of  the  new  enrollees  were  in 
these  lower  grades  ( p  <  .01),  and  a  smaller  percentage  of  the  national  sample  was  represented  by 
these  lower  grades  ( p  <  .05)  when  compared  with  the  regular  participants  in  the  program.  In 
addition,  there  were  a  larger  percentage  of  twelfth  graders  in  the  national  sample  than  in  the 
regular  participant  sample  ( p  <  .001).  Sixty-six  percent  of  the  regular  program  participants  were 
White,  23.1%  were  Hispanic,  7.9%  were  Black,  and  2.8%  were  of  other  races.  The  new  enrollees 
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had  a  racial  distribution  similar  to  that  of  the  regular  program  participants.  Similarly,  the  largest 
majority  of  respondents  in  the  national  sample  was  White  (71%);  however,  the  racial  distribution 
of  the  national  sample  was  different  from  the  regular  participant  sample.  The  national  sample 
was  more  likely  to  be  White  (p  <  .05),  Black  (p  <  .01),  and  of  other  races  (p  <  .001),  and  less 
likely  to  be  Hispanic  ( p  <  .001)  compared  to  the  regular  participants.  As  previously  noted,  the 
regular  participant  sample  was  then  standardized  to  reflect  the  grade,  gender,  and  racial 
distribution  of  the  national  sample  for  subsequent  comparisons. 


Insert  Table  1  here 


Comparison  of  Regular  Participants  and  National  Sample 

Table  2  shows  that  the  weighted  regular  participants  were  significantly  less  likely  than 

the  national  sample  to  use  alcohol  in  the  last  30  days  among  those  respondents  who  had  ever 

2  2 

drank  alcohol  ( X  =  7.18,  p  <  .01)  and  less  likely  to  report  ever  having  an  alcoholic  beverage  (X 

=  28.9,  p  <  .001).  There  were  no  significant  differences  in  cigarette  smoking  among  the  groups. 

2 

Regular  participants  were  about  half  as  likely  to  report  any  illicit  drug  use  for  the  last  30  days  (X 
=  27 .3,  p  <  .001),  and  were  less  likely  to  report  lifetime  use  of  any  illicit  drug  compared  with  the 
national  sample  (A2  =  40.9,  p  <  .001)  (Table  2). 

Because  the  weighting  procedure  affected  the  drug  use  estimates,  an  exploratory  analysis 
was  conducted  using  the  unweighted  regular  participant  data  to  examine  how  this  might  affect 
the  primary  drug  use  outcomes.  Regular  participants  in  the  unweighted  group  were  still 
significantly  less  likely  to  use  any  illicit  drug  in  the  past  30  days  and  in  their  lifetime  than  the 
national  sample  respondents  (p<.0 1  for  30-day  and  lifetime).  In  addition,  due  to  the  large 
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difference  in  sample  size  among  the  comparison  groups,  which  could  potentially  artificially 
increase  precision,  an  exploratory  analysis  on  the  drug  use  variables  was  conducted  using  a 
smaller  matched  comparison  group  from  the  national  database.  We  selected  a  matched  control 
group  of  2  controls  per  case  using  frequency  matching  on  the  proportions  of  gender,  race,  and 
grade  from  the  national  database  (N  =  636).  Results  from  these  comparisons  indicated  that  the 
regular  program  participants  had  significantly  lower  past  30  days  and  lifetime  any  illicit  drug  use 
(/;<.()()  I  for  both  outcomes).  While  constructing  a  matched  data  set  for  all  outcomes  was  not 
deemed  feasible,  the  results  of  this  exploratory  analysis  on  the  main  drug  use  variables  using  a 
smaller  matched  comparison  group  were  consistent  with  the  findings  using  the  full  national 
sample. 

Table  2  shows  that  regular  participants  in  the  youth  development  program  had  more 
drug-averse  attitudes  than  the  national  sample.  These  attitudes  include  higher  personal 
disapproval  of  drug  use,  t(3,946)  =  5.30,  p  <  .001;  greater  perception  of  the  risk  of  harm  from 
drug  use,  /( 26,244)  =  9.90.  p  <  .001;  lower  estimate  of  the  number  of  friends  who  use  drugs, 
t{ 26,685)  =  -5.50,  p  <  .001;  and  greater  difficulty  in  accessing  drugs,  t(30,81 1)  =  -6.19,  p  <  .001 
compared  with  the  national  sample. 

Regular  program  participants  scored  higher  on  the  self-esteem  scale  than  respondents  in 
the  national  sample,  t{  16,099)  =  5-3  ,p<  .01,  indicating  that  program  participants  were  more 
likely  to  respond  positively  to  the  self-confidence  and  self-worth  items  (Table  2). 


Insert  Table  2  here 


Comparison  of  Regular  Participants  and  New  Enrollees 
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Similar  comparisons  showed  that  the  only  measure  on  which  regular  program  participants 
significantly  differed  from  new  enrollees  in  the  program  was  drug  knowledge.  Regular 
participants  in  the  program  achieved  a  higher  score  on  the  drug  knowledge  test  than  did  the  new 
enrollees  to  the  program  ( M  (SD)  =  1 1.0  (2.32)  for  regular  participants  and  M  (SD)  =  10.4  (2.45) 
for  new  enrollees;  t(491)  =  2.53,  p  <  .05).  There  were  no  significant  differences  in  30-day  or 
lifetime  use  of  alcohol,  cigarettes,  and  any  illicit  drug  use  among  regular  participants  and  new 
enrollees.  Similarly,  there  were  no  differences  found  in  any  of  the  drug-related  attitudes  among 
the  two  groups.  Additionally,  regular  participants  and  new  enrollees  had  comparable  scores  for 
self-esteem  and  self-efficacy  ( p  >  .05  for  all  outcomes). 

Pretest-Posttest  Study 

Response  rate  and  analysis  of  nonresponders .  A  total  of  145  follow-up  survey  invitations 
were  sent  to  new  enrollees  who  participated  in  the  initial  survey.  Three  targeted  follow-up 
participants  were  unavailable  due  to  insufficient  or  outdated  contact  infonnation;  therefore,  the 
final  number  of  targeted  follow-up  participants  was  142.  A  total  of  59  posttest  surveys  were 
completed  and  returned,  resulting  in  a  41.5%  overall  posttest  response  rate.  Of  the  59  completed 
posttests  returned,  3  surveys  contained  infonnation  that  was  not  useable,  and  therefore  were  not 
included  in  the  final  follow-up  analysis. 

To  detennine  whether  the  final  follow-up  participant  group  (n  =  56)  and  the 
nonresponder  group  (n  =  93)  differed  on  key  baseline  characteristics,  Pearson  chi-square 
conelation  analyses  and  independent  sample  t  tests  were  conducted  on  several  demographic  and 
drug  use  variables.  Follow-up  study  participants  and  follow-up  nonresponders  did  not 
significantly  differ  in  age,  gender,  race,  grade  level,  or  past  30-day  drug  use;  however,  follow-up 
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participants  were  far  less  likely  to  have  reported  lifetime  use  of  any  illicit  drug  (p  <  .01)  when 
compared  with  nonresponders. 

Findings.  Overall  self-esteem  increased  significantly  among  new  participants  during  the 
first  3  months  of  the  program,  /(44)  =  -2.03,  p  <  .05  (Table  3).  However,  there  were  no 
significant  differences  observed  in  substance  use  behavior,  drug-related  attitudes,  or  self-efficacy 
during  the  study  period.  The  assessment  of  drug  knowledge  unexpectedly  indicated  a  decrease  in 
average  knowledge  scores  over  time  (t( 55)  =  4.74,  p  <  .001).  On  average,  participants  incorrectly 
answered  two  additional  items  at  posttest  compared  with  pretest. 


Insert  Table  3  here 


Discussion 

The  purpose  of  this  evaluation  was  to  detennine  the  effect  of  a  youth  development 
program  on  youth  substance  use.  The  results  indicated  that  program  participants  use  substances 
somewhat  less  and  have  more  drug-averse  attitudes  than  a  national  sample  of  comparable  youth, 
suggesting  that  the  program  successfully  promotes  a  drug-free  lifestyle  among  its  members. 
Alternatively,  it  is  possible  that  these  associations  may  indicate  a  self-selection  bias  into  the 
program.  The  program  may  attract  youth  who  already  have  lower  substance  use  and  drug-averse 
attitudes  than  the  youth  in  the  national  sample.  When  program  participants  were  compared  with  a 
group  of  peers  just  joining  the  program,  there  were  no  differences  observed  in  substance  use 
behavior  and  attitudes;  however,  the  participants  did  demonstrate  higher  drug  knowledge 
compared  with  the  new  program  enrollees.  The  difference  in  the  pattern  of  results  of  the  two 
series  of  comparisons  illustrates  that  there  may  have  been  fundamental  differences  between  the 
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youth  drawn  to  this  program  and  average  youth  in  the  nation.  The  small  difference  in  drug 
knowledge  scores  among  the  more-similar  participant  and  new  enrollee  samples  suggests  that 
regular  membership  in  the  program  does  contribute  to  increased  learning  of  drug  facts. 

Three-month  posttest  results  revealed  an  increase  in  self-esteem  among  new  participants. 
This  increase  in  self-esteem,  which  may  be  the  result  of  how  the  program  teaches  members  about 
respect  for  self  and  a  positive  self-image,  is  an  important  outcome  for  a  broad  youth  development 
program.  An  increase  in  self-esteem  has  significance  in  drug  use  prevention  because  it  has  been 
shown  to  be  associated  with  lower  drug  use  (Emery,  McDermott,  Holcomb,  &  Marty,  1993; 
Newcomb,  Maddahian,  &  Bentler,  1986).  Furthermore,  previous  research  has  shown  that  youth 
with  consistently  low  or  steadily  declining  self-esteem  are  more  susceptible  to  alcohol  use  and 
misuse  (Zimmerman,  Copeland,  Shope,  &  Dielman,  1997). 

The  decrease  in  overall  knowledge  scores  during  the  first  3  months  of  the  program  is  an 
unexpected  result.  This  decrease  may  be  the  consequence  of  a  weakened  ability  to  recall  drug 
education  infonnation.  The  theory  of  context-dependent  memory  posits  that  items  may  be  easier 
to  recall  in  the  same  context  in  which  they  were  learned  (Murnane,  Phelps,  &  Malmberg,  1999). 
Likewise,  information  may  be  more  difficult  to  recall  when  in  a  different  setting  from  that  which 
the  infonnation  was  learned.  It  is  possible  that  since  all  of  the  posttests  were  completed  in  home 
settings,  or  settings  other  than  the  regular  classroom  environment  where  the  drug  information 
was  taught,  recall  may  have  been  challenging  during  the  posttest.  Nonetheless,  this  finding 
supports  the  recommendation  to  intensify  drug  use  prevention  training  during  the  first  quarter  for 
new  members.  It  also  points  to  the  need  for  more  aggressive  and  interactive  methods  of  teaching 
drug  abuse  prevention  lessons. 
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The  lack  of  significant  changes  in  drug  use  behavior  during  the  short  study  period 
suggests  that  the  program  was  not  sufficient  to  produce  short-term  behavioral  improvements. 
However,  it  is  important  to  consider  that  health  behavioral  change  is  often  difficult  to  achieve, 
even  with  the  most  comprehensive  of  interventions.  It  is  likely  that  the  3-month  study  period  was 
not  sufficiently  long  enough  for  behavioral  and  attitudinal  change  to  occur,  or  for  adequate 
detection  of  those  changes.  Furthennore,  the  present  study  did  not  measure  all  of  the  possible 
positive  outcomes  that  may  have  been  affected  by  the  program,  such  as  resilience,  emotional 
competence,  or  school  retention. 

Several  limitations  of  the  present  study  should  be  noted.  The  comparison  data  are  cross- 
sectional  and  were  used  to  examine  associations  rather  than  causation.  The  cross-sectional 
approach  cannot  provide  any  direct  insight  into  changes  in  drug  use  and  related  factors  that  may 
have  occurred  over  time.  However,  to  provide  a  context  within  which  to  evaluate  the  cross- 
sectional  data,  we  compared  our  assessment  with  a  national  sample,  as  well  as  new  program 
enrollees.  The  use  of  a  one-group  pretest-posttest  design  for  another  component  of  the  study  is 
another  limitation  due  to  the  threats  to  validity  associated  with  this  design.  However,  no  viable 
control  group  was  available  and  random  assignment  to  groups  was  not  feasible  for  evaluating  an 
existing  program.  Furthermore,  the  consistency  of  program  implementation  and  its  intensity 
during  the  study  period  were  not  assessed.  As  previously  noted,  the  nonresponder  analysis  for  the 
pretest-posttest  study  indicated  that  follow-up  participants  were  less  likely  than  dropouts  to  have 
reported  lifetime  use  of  any  illicit  drug.  This  indicates  that  generalization  of  the  study  results  to 
youth  with  higher  lifetime  drug  use  may  be  limited.  Also,  the  follow-up  sample  size  was  small, 
which  can  make  it  more  difficult  to  identify  statistical  trends  (Kish,  1965;  Last,  1988;  Sackett, 
1979);  however,  our  follow-up  participation  rate  of  41%  was  comparable  to  average  response 
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rates  achieved  in  other  Web-based  and  mailed  surveys  (Shih  &  Fan,  2008).  It  should  also  be 
noted  that  while  most  of  the  survey  measures  were  validated  and  used  extensively  in  previous 
research,  they  were  all  self-reported,  therefore  bias  may  have  been  introduced  during  data 
collection.  To  reduce  subjective  influence  or  pressure  to  respond  in  a  socially  desirable  manner, 
adult  leaders  and  parents  were  not  permitted  in  the  room  during  survey  administration,  and 
researchers  reminded  participants  about  the  confidentiality  of  their  data. 

This  study  compared  drug  use  and  related  factors  of  participants  in  a  youth  development 
program  with  two  peer  groups  and  examined  short-term  changes  among  new  participants.  The 
results  of  this  study  indicate  some  differences  in  drug  use  and  related  factors  among  participants 
and  a  national  sample,  and  a  small  difference  in  drug  knowledge  among  participants  and  new 
enrollees.  The  study  also  showed  that  the  program  may  contribute  toward  the  positive 
development  of  participants’  self-esteem.  Other  results  and  a  lack  of  significant  findings 
highlight  areas  where  improvements  can  be  made  to  strengthen  the  program.  The  short-term 
decrease  in  drug  knowledge  found  in  this  study  indicates  that  there  is  a  need  for  stronger  efforts 
to  help  new  participants  acquire  and  maintain  drug  prevention  knowledge  early  in  their 
participation  in  the  program.  These  efforts  might  include  the  use  of  incentives  for  participants  to 
pass  drug  knowledge  tests  and  improve  individual  scores,  brief  booster  sessions,  and  interactive 
methods  to  reinforce  basic  drug  knowledge.  Additionally,  because  this  study  found  only  minimal 
effects  and  no  changes  in  drug  use  behavior  over  time,  it  is  recommended  that  the  program  adopt 
a  standardized  drug  prevention  educational  curriculum  to  increase  the  efficacy  and  consistency 
of  the  program.  The  use  of  a  standardized  curriculum  is  a  consistent  feature  of  effective  youth 
development  programs  (Catalano  et  al.,  2002)  and  would  likely  lead  to  more  substantial 
improvements  in  drug-related  behaviors  and  attitudes  among  participants.  While  the  present 
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program  provides  positive  influences  through  a  variety  of  activities  and  direct  interaction  with 
the  leaders,  adherence  to  structured  lessons  plans  will  likely  provide  consistent  opportunities  for 
the  practice  of  the  program’s  drug  use  prevention  strategies  and,  in  turn,  advancements  in  drug 
use  prevention  among  participants. 
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Table  1. 

Demographic  Characteristics  of  Youth  Development  Program  Regular  Participants  and 
Comparison  Groups 


Variable 

Regular 

participants 

Comparison  groups 

New  enrollee 
National  sample3  participants 

Age  in  years,  M  (SD) 

13.9(1.9) 

b 

13.4(1.6) 

n 

360 

— 

144 

Gender  (%  male) 

__  _*** 

77.3 

48.4 

83.3 

n 

361 

46,466 

144 

Grade  (%) 

6th-8th 

54.5  T 

35.2 

74.6 

9th- 10th 

30.0 

34.9 

18.3 

11th- 12th 

15.6*** 

29.9 

7.0 

n 

347 

45,466 

142 

Race  (%) 

Black 

7.9 

13.7 

11.9 

Hispanic 

23.1 

15.3 

26.3 

White 

66.2* 

71.0 

58.5 

Otherc 

*** 

2.8 

12.5 

3.4 

n 

355 

46,466 

118 

Note :  M  =  mean;  SD  =  standard  deviation. 


Differs  from  national  sample,/)  <  .05. 

^Differs  from  national  sample,  p  <  .01. 

***Differs  from  national  sample,/)  <  .001. 

'Differs  from  new  enrollee  participants,  p  <  .01. 

aMonitoring  the  Future  (MTF)  series,  2006  data,  Substance  Abuse  and  Mental  Health  Data 
Archive. 

bA  continuous  age  variable  was  not  included  in  the  publicly  available  MTF  data. 

cOther  race  includes  Asian,  American  Indian  or  Alaska  Native,  or  Native  Hawaiian  or  Other 


Pacific  Islander. 
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Table  2. 

Comparison  of  Youth  Development  Program  Regular  Participants  With  a  National  Sample  on 
Substance  Use,  Drug-Related  Attitudes,  and  Self-Esteem 


Weighted 

regular  participants3 

National  sampleb 

Variable 

M(SD)  or  % 

n 

M(SD)  or  % 

n 

Substance  use 

Alcohol  (%) 

30-day  usec 

38.6 

130 

50.4 

15,652 

Lifetime  use 

** 

37.1 

373 

51.1 

42,297 

Cigarette  smoking  (%) 

30-day  use 

35.6 

107 

41.6 

15,188 

Lifetime  use 

28.8 

374 

35.4 

45,663 

Any  illicit  drug  use  (%) 

30-day  use 

10.0 

346 

21.6 

34,204 

Lifetime  use 

26.7** 

359 

43.5 

38,038 

Drug-related  attitudes,  M  (SD) 

Personal  disapproval6 

2.57(0.41) 

54* 

2.15  (0.58) 

3,894s 

Risk  of  harm  of  drug  useh 

3.47  (0.69)** 

362 

3.10(0.70) 

25,884 

Friends’  drug  use1 

1.87  (0.82)** 

371 

2.12(0.87) 

26,316 

Access  to  drugs’ 

3.42  (1.43)** 

361 

3.85  (1.31) 

30,452 

Self-esteem,  M  (SD)k 

4.2  (0.8)  * 

349 

4.0  (0.7) 

15,752 

Note :  M  =  mean;  SD  =  standard  deviation. 


p  <  .01. 

*><.001. 

aRegular  program  participants  were  weighted  on  gender,  grade,  and  race  to  represent  the 
distribution  of  the  national  sample. 

bMonitoring  the  Future  (MTF)  series,  2006  data,  Substance  Abuse  and  Mental  Health  Data 
Archive. 

c30-day  alcohol  use  was  reported  among  only  those  participants  who  had  ever  drank  alcohol. 
d3  0-day  cigarette  use  was  reported  among  only  those  participants  who  had  ever  smoked 
cigarettes. 

e6-item  scale;  response  options:  1  =  Don’t  disapprove,  3  =  Strongly  disapprove. 

The  personal  disapproval  scale  was  computed  for  only  1 1th  and  12th  graders  in  the  weighted 
regular  participant  sample  to  be  comparable  to  the  national  sample  data. 
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gPersonal  disapproval  scale  items  were  included  only  for  a  subset  of  the  12th  grade  version  of  the 
2006  MTF  survey. 

h6-item  scale;  response  options:  1  =  No  risk,  4  =  Great  risk. 

'S-item  scale;  response  options:  1  =  None,  5  =  All. 

J2-item  scale;  response  options:  1  =  Probably  impossible,  5  =  Very  easy. 
k12-item  scale;  response  options:  1  =  Disagree,  5  =  Agree. 
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Table  3. 

Substance  Use,  Drug-Related  Attitudes,  Knowledge,  and  Psychosocial  Characteristics  at  Pretest 
and  Posttest  Among  Youth  Development  Program  New  Enrollee  Participants 


Variable 

n 

Pretest 

M  (SD)  or  % 

Posttest 

M  (SD)  or  % 

Substance  use  (%) 

30-day  alcohol  use3 

7 

57.1 

42.9 

30-day  cigarette  smokingb 

10 

50.0 

40.0 

30-day  any  illicit  drug  use 

50 

8.0 

8.0 

Drug-related  attitudes,  M  (SD) 

Personal  disapproval3 

48 

2.61  (0.49) 

2.67  (0.44) 

Risk  of  harm  of  drug  used 

45 

3.48  (0.43) 

3.60  (0.60) 

Friends’  drug  use6 

47 

1.61  (0.88) 

1.76  (0.77) 

Access  to  drugs1 

49 

2.70(1.51) 

2.53  (1.60) 

Drug  knowledge,  M  ( SD)s 

56 

10.71  (1.92) 

8.21  (3.22)** 

Psychosocial  characteristics,  M 
(SD) 

Self-esteem 

45 

4.2  (0.8) 

4.4  (0.8) 

Self-efficacy1 

45 

3.6  (0.5) 

3.6  (0.4) 

Note :  M  =  mean;  SD  =  standard  deviation. 

><.05. 

*  * 


a30-day  alcohol  use  was  reported  among  only  those  participants  who  had  ever  drank  alcohol. 
b3  0-day  cigarette  use  was  reported  among  only  those  participants  who  had  ever  smoked 
cigarettes. 

c6-item  scale;  response  options;  1  =  Don’t  disapprove,  3  =  Strongly  disapprove. 
d7-item  scale;  response  options;  1  =  No  risk,  4  =  Great  risk. 
e5-item  scale;  response  options;  1  =  None,  5  =  All. 

2-item  scale;  response  options;  1  =  Probably  impossible,  5  =  Very  easy. 

gNumber  of  correctly  answered  drug  knowledge  items;  possible  maximum  score  of  15. 

h  12-item  scale;  response  options;  1  =  Disagree,  5  =  Agree. 

‘10-item  scale;  response  options;  1  =  Strongly  agree,  4  =  Strongly  disagree. 
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